Skip to content

Add token refresh and Argon2 secret key support#83

Open
jlegrand62 wants to merge 41 commits intodevfrom
hotfix/jwt_exp_date
Open

Add token refresh and Argon2 secret key support#83
jlegrand62 wants to merge 41 commits intodevfrom
hotfix/jwt_exp_date

Conversation

@jlegrand62
Copy link
Member

@jlegrand62 jlegrand62 commented Feb 4, 2026

Summary

This pull request introduces comprehensive authentication enhancements across the PlantDB client and server:

  • Token refresh flow

    • New /refresh endpoint on the REST API that validates a refresh token and returns a new access/refresh token pair.
    • Client-side helpers (request_token_refresh, _request_with_refresh, token_refresh_url) automatically handle 401 responses and retry with a refreshed token.
    • Updated login, logout, and all protected API calls to use the new refresh‑aware request logic.
  • JWT session manager upgrades

    • Added refresh_timeout, leeway, and thread‑safe lock handling.
    • Session creation now returns both access and refresh tokens.
    • Explicit validation errors (SessionValidationError, RefreshTokenReuseError, etc.) and UTC timestamps.
    • Support for maximum concurrent sessions per user.
  • Argon2‑derived secret key

    • Introduced _derive_key_argon2 and _init_secret_key helpers to generate a 64‑byte HS512 key from a pass‑phrase or random bytes.
    • Updated JWTSessionManager to use the new secret‑key initialization.
  • Configuration & security

    • Increased default entropy for FLASK_SECRET_KEY and JWT_SECRET_KEY to 64 bits.
    • Added environment variables SESSION_TIMEOUT, REFRESH_TIMEOUT, and MAX_SESSION with sensible defaults.
    • Refactored secret‑key handling to use the shared _init_secret_key utility.
  • API updates & fixes

    • New API endpoints: scans_info, create_scan, create_fileset, create_file, list_scan_filesets, list_fileset_files, and metadata endpoints for scans/filesets/files.
    • Image endpoint now accepts as_base64 flag; related URL helpers (scan_image_url, list_task_images_uri, get_image_uri) propagate this flag and build proper query strings.
    • Refactored client (PlantDBClient) to store _access_token / _refresh_token, use _request_with_refresh, and provide validate_token, refresh_token, and improved logout handling.
    • Added token‑validation and token‑refresh URL helpers plus request functions (request_token_validation, request_token_refresh).
  • Code quality & documentation

    • Reformatted docstrings, function signatures, and spacing for consistency.
    • Added module‑level logger and improved error logging in REST API client.
    • Cleaned up stray debug prints and updated examples throughout the repository.
    • Expanded unit tests for session management and JWT handling.
  • Miscellaneous

    • Adjusted pyproject.toml requirements (requires-python upper bound) and removed unused test dependencies.
    • Updated tests to use the new token fields and to exercise the new authentication flow.

These changes provide robust token lifecycle management, stronger cryptographic defaults, and clearer, more maintainable code.

- Define new environment variables for JWT session timeout (default 3600s) and max concurrent sessions (default 10).
- Pass these values to `JWTSessionManager` when initializing the local FSDB.
- Replace `_get_env_secret` with `os.getenv` for `SESSION_TIMEOUT` and `MAX_SESSION` in `fsdb_rest_api.py`
- Apply the change consistently in both temporary test database setup and main FSDB initialization.
- Add `token_validation_url()` to construct the `/token-validation` URL
- Add `request_token_validation()` to POST to the endpoint and return the response
- Update docstrings and examples for the new helpers
- Implements a helper that POSTs to the `/token-validation` endpoint
- Returns a boolean indicating if the JWT token is still valid
- Handles network or unexpected errors gracefully
- Replace “JWT token” with “JSON Web Token” in `src/server/plantdb/server/rest_api.py`
- Update corresponding client documentation in `src/client/plantdb/client/plantdb_client.py` to reference “JSON Web Token” instead of “JWT token”
- Add `-> str` return type annotations to `sanitize_name` and `create_user` in `src/client/plantdb/client/api_endpoints.py`
- Introduce `scans_info` helper returning `"/scans_info"` with full docstring
- Add creation helpers `create_scan`, `create_fileset`, `create_file` returning respective `"/api/scan"`, `"/api/fileset"`, `"/api/file"` paths
- Add listing helpers `list_scan_filesets` and `list_fileset_files` with sanitized parameters and URL construction
- Add metadata helpers `metadata_scan`, `metadata_fileset`, and `metadata_files` for accessing scan, fileset, and file metadata
- Apply `@url_prefix` decorator to all new endpoint functions
- Expand docstrings for the new functions to include parameters, returns, and usage examples
- Change docstring default description for `fuzzy` to ``False`` (capitalized) in `src/server/plantdb/server/rest_api.py`
- Refactor example code
- Adjust accompanying comments to reflect the updated example behavior.
- Change `plantdb_url()` call to `plantdb_url('localhost', port=5000)` in the basic usage example (`src/client/plantdb/client/plantdb_client.py`).
- Update test server example to use port `5000` instead of `5555` and adjust subsequent `PlantDBClient` instantiations accordingly.
- Import `Type` and `hash_secret_raw` from **argon2** in `src/commons/plantdb/commons/auth/session.py`
- Add static method ``_derive_key_argon2`` to stretch a pass‑phrase into a 64‑byte HS512 key
- Add ``_init_secret_key`` to handle `secret_key` initialization:
  - Generate random 64‑byte key when ``None``
  - Validate length for ``bytes`` input
  - Derive key with Argon2 for ``str`` input
- Change ``__init__`` signature to ``secret_key: Union[bytes, str]`` and invoke ``_init_secret_key``
- Update docstring for ``secret_key`` to describe accepted types and defaults
- Ensure ``_create_token`` returns a string by decoding the JWT bytes when needed
- Adjust related imports and type hints accordingly
…dling, and explicit validation errors

- Introduced custom exception hierarchy (`SessionValidationError`, `AccessTokenNotFoundError`, `RefreshTokenNotFoundError`, `InvalidTokenProcessingError`, `RefreshTokenReuseError`, `WrongTokenType`) in `src/commons/plantdb/commons/auth/session.py`
- Imported `timezone` and `RLock` to enable UTC timestamps and thread‑safe session modifications
- Updated `SessionManager`:
  - All datetime operations now use `datetime.now(timezone.utc)`
  - Return types refined to `Union[dict, None]`
  - `invalidate_session` now returns `Union[str, None]`
  - Cleanup uses UTC and lock protection
- Extended `JWTSessionManager`:
  - Added `refresh_timeout`, `leeway`, `_lock`, and `refresh_tokens` tracking
  - Constructor signature now includes `refresh_timeout` and `leeway`
  - `_create_token` now accepts `token_type` and adds a `type` claim
  - `create_session` generates and returns a tuple of `(access_token, refresh_token)` and records both access and refresh tokens under the lock
  - `validate_session` now validates token type, raises specific `SessionValidationError` subclasses, and updates last‑access timestamps thread‑safely
  - `invalidate_session` handles removal of both access and linked refresh tokens
  - `cleanup_expired_sessions` removes expired access and refresh entries
  - `session_username` catches validation errors and logs a warning
  - `refresh_session` validates the refresh token, rotates both access and refresh tokens, and returns a new token pair
- Updated docstrings and logging messages to reflect new behavior and terminology (e.g., “JSON Web Token”)
- Introduce `src/commons/tests/test_auth_session.py` with test cases for `SessionManager`
  - Verify initialization, session creation, validation, expiration, cleanup, username lookup, and invalidation
- Add extensive `TestJWTSessionManager` suite
  - Test secret key handling, token generation, payload contents, and storage of JTI values
  - Validate access and refresh token processing, including timestamp updates and type checks
  - Cover error scenarios: token type mismatch, expiration, malformed tokens, wrong audience/issuer, missing tokens
  - Test invalidation paths for access and refresh tokens and handling of unknown JTI
  - Verify refresh flow rotation, reuse detection, and cleanup of expired entries
  - Ensure max concurrent session limit enforcement
  - Test `session_username` helper for valid and invalid tokens
- Import `SessionValidationError` in `src/server/plantdb/server/rest_api.py` and return **401** with “Invalid credentials” for user creation, logout, scan creation, metadata updates, fileset/file operations, and other protected actions.
- Update login flow to use `self.db.login` returning a tuple `(access_token, refresh_token)` and include both tokens in the successful response.
- Refine logout handling to check the success flag from `self.db.logout` and return appropriate **200** or **401** messages.
- Rewrite token refresh endpoint:
  - Remove the JWT header decorator.
  - Expect a JSON payload containing `refresh_token`.
  - Validate the refresh token via `self.db.session_manager.refresh_session`.
  - Return a new pair of `access_token` and `refresh_token` or appropriate error responses.
- Add permission error handling when creating a scan, returning **401** on `PermissionError`.
- Fix typo in docstring (“you” instead of “uou”) and improve example comments.
- Ensure all new error paths consistently return “Invalid credentials” for session validation failures.
- Update comment in `test_rest_api_server.py` to clarify that the first request is made without a login
- Remove stray `print(info)` debug output from the metadata test
- Improve comment wording about resetting the file pointer before the second request
- Minor comment phrasing adjustment for consistency
- Delete `print(f"Received: {bytes_received}")` from `write_stream` in `src/server/plantdb/server/rest_api.py` to clean up console output.
…g in JWT session manager

- Introduce `_derive_key_argon2` in `src/commons/plantdb/commons/auth/session.py` to stretch a pass‑phrase into a 64‑byte HS512 key using Argon2.
- Add `_init_secret_key` utility that generates a random 64‑byte key, validates provided byte keys, or derives a key from a string via `_derive_key_argon2`.
- Update `JWTSessionManager.__init__` to initialize `self.secret_key` with `_init_secret_key`.
- Remove the previous static `_derive_key_argon2` method and the instance `_init_secret_key` implementation; replace them with the new module‑level functions.
- Adjust the class to call the shared `_init_secret_key` helper.
- Add required imports `Type` and `hash_secret_raw` from **argon2**.
- Increase default entropy for `FLASK_SECRET_KEY` and `JWT_SECRET_KEY` to 64 bits in `src/server/plantdb/server/cli/fsdb_rest_api.py`.
- Change `SESSION_TIMEOUT` default to `900` seconds (15 min) and introduce new `REFRESH_TIMEOUT` default of `86400` seconds (1 day).
- Document the new defaults in the file’s usage section.
- Import `_init_secret_key` and replace direct random generation with `_init_secret_key` for secret handling.
- Read `REFRESH_TIMEOUT` from the environment and pass it to `JWTSessionManager`.
- Extend `JWTSessionManager` initialization with the `refresh_timeout` argument.
…WSGI CLI

- Update `FLASK_SECRET_KEY` and `JWT_SECRET_KEY` defaults from 32‑bit to 64‑bit random values in `src/server/plantdb/server/cli/wsgi.py`
- Document new environment variables:
  - `SESSION_TIMEOUT`: JWT session validity (default `900` seconds)
  - `REFRESH_TIMEOUT`: JWT refresh token validity (default `86400` seconds)
  - `MAX_SESSION`: maximum concurrent sessions per user (default `10`)
- Extend usage section to reflect the added defaults and descriptions.
- Adjust spacing in type annotations for `parsing`, `_get_env_secret`, and `_configure_app` in `src/server/plantdb/server/cli/fsdb_rest_api.py`
- Move opening triple‑quote to the same line as the docstring content for these functions
- Apply minor whitespace cleanup for consistent style across the file
…ent.py`

- Import `json` for query serialization.
- Rename public JWT attributes to private fields: `jwt_token` → `_access_token`, add `_refresh_token` and `_username`.
- Remove the old `validate_session_token` method.
- Introduce `_request_with_refresh` to automatically refresh the access token on a 401 response.
- Update `login` to store both access and refresh tokens and to use `session.request` directly.
- Refactor `logout`, `create_user`, `refresh`, `list_scans`, `list_scans_info`, `create_scan`, metadata and fileset operations to use `_request_with_refresh`.
- Add `validate_token` method for token validation via the refreshed request helper.
- Simplify `refresh_token` to use stored refresh token, update headers, and handle errors.
- Revise `_handle_http_errors` to log severity‑based messages and raise a generic `RequestException`.
- Add new `list_scans_info` API wrapper returning detailed scan dictionaries.
- Update docstrings and examples to reflect new attribute names and authentication flow.
- Refactor `origin_url` in `src/client/plantdb/client/rest_api.py` to use `urllib.parse` (`urlparse`, `urlunparse`, `splitport`) with comprehensive docstring, input validation, proper scheme detection, and clean port handling.
- Introduce module‑level logger via `get_logger(__name__)`.
- Add `token_refresh_url` helper to build the full token‑refresh endpoint URL.
- Add `request_token_refresh` function to perform a POST request to the refresh endpoint, returning the API response.
- Update `make_api_request` to log SSL and request errors with the logger, normalize HTTP method name, and improve header handling for `session_token`.
- Extend docstrings and examples for `make_api_request`, `request_token_refresh`, and related functions.
- Move multiline docstrings to a single line directly after the definition in `src/client/plantdb/client/url.py`, `src/client/plantdb/client/api_endpoints.py` and `src/client/plantdb/client/sync.py`
- Add a space before the colon in return‑type annotations (e.g. `-> Optional[set[str]] :`) across the updated functions
- Apply the same inline‑docstring style and signature spacing to class definitions in `src/commons/plantdb/commons/fsdb/lock.py` and `src/commons/plantdb/commons/auth/rbac.py`
- Remove unused `secrets` import
- Add a space before the colon in return‑type annotations for `parsing`, `_get_env_secret`, and `_configure_app`
- Move opening triple‑quote to the same line as the docstring content for these functions
- Insert spaces around assignment operators for `session_timeout`, `max_sessions`, and `refresh_timeout`
- Change return type annotation from `Flask` to `flask.Flask` in `_configure_app`
- Update `app` parameter annotation to `flask.Flask` for consistency
@jlegrand62 jlegrand62 added the enhancement New feature or request label Feb 4, 2026
- Update docstring in `src/server/plantdb/server/rest_api.py` to use ``as_base64`` instead of ``base64`` and improve description wording.
- Change query‑parameter parsing from ``request.args.get('base64'…)`` to ``request.args.get('as_base64'…)`` and rename the flag variable to ``as_base64``.
- Adjust related comments and conditional logic to reference ``as_base64``.
- Refine the “See Also” section wording for consistency.
- Import `urllib.parse` as `parse` in `src/client/plantdb/client/api_endpoints.py`.
- Extend `image` signature to `def image(..., as_base64: bool, **kwargs)`.
- Update docstring to describe the new `as_base64` flag and examples.
- Build query string with `parse.urlencode`, handling `size` and `as_base64` flags, and return the assembled URL.
- Refactor `sequence` to construct its query string via a `query` dict and `parse.urlencode`, preserving the existing `type` parameter.
- Extend `scan_image_url` signature in `src/client/plantdb/client/rest_api.py` with `as_base64=False`.
- Document the new `as_base64` parameter in the function docstring.
- Forward `as_base64` to `api_endpoints.image` when building the image URL.
- Update `self.db.logout` call in `src/server/plantdb/server/rest_api.py` to capture both `success` and `username` (`success, username = self.db.logout(**kwargs)`).
- Return a success message that includes the logged‑out user: `{'message': f'Logout successful from {username}'}`.
- Refine failure and error messages across the endpoint:
  - Use `'Logout failed!'` for generic failures.
  - Use `'Logout failed, no active session!'` when no session token is provided.
  - Append exclamation marks to `'Invalid credentials!'` and `'Logout failed!'` responses for consistency.
jlegrand62 and others added 13 commits February 4, 2026 23:48
- Update return type annotation to `tuple[bool, str]` in `src/commons/plantdb/commons/fsdb/core.py`.
- Modify docstring example to show `(True, 'admin')` as the successful result.
- Adjust implementation to return `(success, username)` for both success and failure cases.
- Import `Union` for type hints.
- Extend `list_task_images_uri` with `as_base64` flag and update URL construction.
- Update `request_login` to return a `dict` by calling `.json()`.
- Change `request_check_username` to return a `bool` (`exists` field).
- Refactor `request_logout` to return `(bool, str)` with success flag and message.
- Make `request_token_validation` and `request_token_refresh` return parsed `dict` objects.
- Adjust `request_new_user` to return a `bool` indicating success.
- Update `request_scan_names_list`, `request_scans_info`, and `request_scan_data` with precise return type annotations.
- Add `as_base64` parameter to `request_scan_image`, returning either raw bytes or a base64‑encoded dict.
- Annotate `request_scan_tasks_fileset` to return a `dict`.
- Modify `request_refresh` to return `(bool, str)` success flag and message.
- Extend `parse_task_images` to accept `as_base64` and forward it to `list_task_images_uri`.
- Unpack `(success, msg)` from `request_refresh(**server_cfg)` in `src/client/plantdb/client/sync.py` example.
- Capture `success, msg = request_refresh(scan_id, host=target_host, port=target_port)` in `src/client/plantdb/client/cli/fsdb_rest_api_sync.py` for proper error handling.
- Add `request_login` calls and use returned `access_token` as `session_token` for all API requests (`request_scan_names_list`, `request_scan_data`, `request_scans_info`, `scan_preview_image_url`, `list_task_images_uri`, `parse_task_images`, `request_refresh`, `get_task_data`)
- Adjust `test_refresh_and_archive` to unpack `(success, message)` from `request_refresh` and assert `success` is `True`
- Update imports and variable handling to reflect new authentication flow throughout the test file.
- Update `requires-python` in `src/server/pyproject.toml`, `src/commons/pyproject.toml` and `src/client/pyproject.toml` to `>=3.8, <3.13` with a comment about the 3.13 upper bound
- Add `open3d>=0.9.0.0` to the server extra dependencies list in `src/server/pyproject.toml`
- Enable `open3d >=0.9.0.0` in the commons core dependencies by uncommenting it in `src/commons/pyproject.toml`
- Remove `open3d` from the client test dependencies in `src/client/pyproject.toml`
- Modify the doctest in `src/client/plantdb/client/rest_api.py` to unpack `(success, msg)` from `request_logout` and print `success` instead of the previous `logout` variable.
- Extend `scan_image_url` in `src/client/plantdb/client/rest_api.py` with an `as_base64` flag and update the doctest to show the new usage.
- Change `request_scan_image` signature to return `tuple[str, str, Union[str, bytes]]` (`content_type`, `encoding`, image data) and revise its documentation and examples accordingly.
- Introduce `wants_base64` static method in `src/server/plantdb/server/rest_api.py` to detect the `as_base64` query parameter.
- Update the image `GET` endpoint to:
  - Return a JSON payload containing `image` (base64 string) and `content-type` when `as_base64` is requested, adding `Content-Type: application/json` and `X-Content-Encoding: base64` headers.
  - Stream the raw image file otherwise, with `X-Content-Encoding: binary` header.
- Import `parse` from `urllib` in `src/server/plantdb/server/rest_api.py`
- Extend `get_image_uri` signature to include `as_base64=False` and update its docstring
- Build query parameters dictionary handling `size` and `as_base64`, encode with `parse.urlencode`
- Return the image path with the assembled query string, preserving existing behavior when no optional parameters are provided
- Update doctest example to reflect the new `as_base64` flag usage
- In `src/client/plantdb/client/plantdb_client.py`, extract `resp_username` from the token validation response.
- Set `self._username` to `resp_username` when it is not already set and immediately refresh the token.
- If `self._username` is set and differs from `resp_username`, log a warning: `Given token correspond to a different username`.
- Introduce `_warning_timestamps` and `_warning_debounce_interval` in `lock.py` to limit repeated warning messages per `scan_id`
- Add configurable `warning_debounce_interval` (default 5 seconds) via constructor kwargs
- Track acquisition attempts with a new `attempt` counter
- Clear stale warning timestamps on successful lock acquisition
- Enhance debug logs to include attempt count and success details
- Emit warning only after debounce interval; otherwise log at DEBUG level to reduce spam
- Update log messages to include process ID, thread ID, and attempt number for better traceability
- Replace all `scan.get_metadata()` calls with `_get_metadata(scan.metadata, None, {})` in `src/commons/plantdb/commons/fsdb/core.py` to prevent nested lock acquisition.
- Update permission checks (`can_access_scan`) to use the directly accessed `metadata` for READ, WRITE, and DELETE operations across scans, filesets, and files.
- Add explanatory comments before each direct metadata access.
- Adjust DELETE permission validation in `remove_scan` to use the new metadata access pattern.
- Modify metadata validation in `update_metadata` to retrieve `old_metadata` via direct access.
- Update error message for missing files to include full `scan.id/fileset.id` path.
- Import `FSDB` and `setup_test_database` in `test_fsdb_lock.py`
- Create temporary FSDB directory in `setUp` using `setup_test_database(['real_plant', 'real_plant_analyzed'])`
- Clean up directory in `tearDown` with `shutil.rmtree`
- Add `TestFSDBLock` suite covering:
  - `test_delete_file_without_permission` – expect `PermissionError` for guest
  - `test_delete_file_with_permission` – admin can delete a file
  - `test_delete_fileset_without_permission` – guest cannot delete a fileset
  - `test_delete_fileset_with_permission` – admin can delete a fileset
  - `test_delete_scan_without_permission` – guest cannot delete a scan
  - `test_delete_scan_with_permission` – admin can delete a scan
- Replace `import:` with `inventories:` under the Python handler in `mkdocs.yml` to correctly reference external object inventories.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant